Multivariate testing (otherwise known as “split testing” or “A/B testing”) is a common method of evaluating changes to website design and content. In a multivariate website test, two or more versions of a particular webpage are created; these are then displayed to randomly selected groups of site visitors. Usually, one of these versions is the current version of the webpage serving as an experimental control. A randomly selected subset of visitors to the website are presented this version, and they are referred to as a “control group.” Randomly selected others of the visitors are randomly presented with another version that implements proposed changes to the current version, and it should be noted that there can be more than one of such other versions in which different proposed changes are implemented. Thus there could be one other group presented with a version implementing proposed changes, or more than one other group in which each of the other groups is presented with a different other version implementing different proposed changes.
The manner in which the visitors in each of the groups responds to the particular version presented to each of them is monitored. In particular, statistical data indicative of a rate at which one or more specific behaviors by visitors occurs in each of the groups are gathered. Typically, there is a desire to increase or decrease a rate of a specific behavior of visitors to a website (e.g., a rate of purchases of products or usage of a given service). The statistical data is subjected to one or more forms of statistical analysis to identify any statistically significant increase or decrease in a given behavior arising from a version of the webpage that implements proposed changes. If a given version implementing proposed changes is found to bring about a statistically significant improvement in the rate with which a specific behavior occurs (keeping in mind that an improvement may be an increase or a decrease in that rate, depending on what the specific behavior is), then that version implementing those proposed changes may be adopted as the new version of the webpage.
Some multivariate tests are simple, e.g., a straightforward A/B test. Many are more complex. An A/B test has one variable with two values; a more complex test may have several variables with several values each. As the number of variables and values increases, the number of web page versions (and correspondingly, the number of visitor test groups needed) increases rapidly. A complex test might require tens or even hundreds of visitor groups.
In multivariate forms of A/B testing in which multiple combinations of variations in content presentation in a user interface are proposed, the work of entering the specific combinations of variations to be tested can become overwhelming and cumbersome. Many currently available systems for specifying the exact proposed combinations of variations in user interface content presentation to be tested employ configuration environments that merely expand on longstanding configuration environments originally created to accommodate specifying only one variation per test, or at most, specifying multiple variations in only one area of content presentation in a user interface per test.
In multivariate user interface content testing, the number of possible combinations of variations of content presentation in a user interface to be tested increases exponentially each time one more area for content presentation and/or one more variation in content presentation is specified to be tested. By way of example, testing just a few variations of text size, with just a few variations of text font, with just a few variations of text color for presentation of textual content can quickly result in an unwieldy list of possible combinations of different text size, font and color that could be tested. It follows that entering such an unwieldy list of possible combinations can, itself, become unwieldy in a configuration environment that is ill suited to enable more graceful entry of so many possible combinations. It is with respect to these and other considerations that the techniques described herein are needed.